An omni-direction image (omni-images) that acquire a skyline of an urban environment by an omni-directional camera (omni-camera) can be used for geo-localization. As an advantage, the representation of the skyline can be compact, is unique, and remarkably informative about a location. The skyline is also less prone to occlusions by trees, lampposts, vehicles, pedestrians, and lighting variations in other visual features. Of particular interest are urban skylines.
The main reason to use omni-images instead of perspective images comes from the fact that the stability of the underlying pose estimation problem increases with the field of view of the camera. Omni-images also can give much better accuracy in motion estimation than conventional perspective images. In the case of small rigid motions, two different motions can yield nearly identical motion fields for conventional images, which is not the case for omni-images.
The basic conventional method matches a query image of an unknown location with database images of known locations. In order to match the query image with a large collection of images features based on color histograms, texton, histograms, line features, gist descriptor, and geometric context can be used.
Some methods can match lines in 3D models with lines in the images. Such geometry-matching methods are generally efficient but have very high sensitivity to imperfections in a feature extraction pipeline.
It is often desired to determine one's location, for example when driving. One way to do this is to use a global positioning system (GPS). Unfortunately, GPS has limitations because the signals are broadcasted at 500 Watts from satellites about 12,000 miles up. Signals from four satellites are required for normal operation. The signals can be obstructed by buildings, and even foliage. This is called the urban canyon problem where radio reception is poor.
Therefore, it is desired to use computer vision techniques. In computer vision applications, images are analyzed to determine poses, i.e., location and orientation. Pose estimation, although extremely challenging due to several degeneracy problems and inaccurate feature matching, is well known in computer vision. However, most conventional solutions are only proven on a small scale, typically in a well controlled environment.
One system uses an infrared camera and a 3D model generated from an aerial laser survey. That system requires an expensive infrared camera, which makes it impractical for large-scale deployment in consumer oriented applications, such as vehicle or hand-held devices. The camera in that system has a restricted field of view. To provide a partial 360° view primary and secondary mirrors are placed directly in the optical path between the scene and camera. The mirrors obscure a large central portion of the infrared images.
Another method is described in U.S. application Ser. No. 12/495,655, “Method for Determining a Location From Images Acquired of an Environment with an Omni-Directional Camera,” filed Jul. 22, 2009 by Ramalingam et al.